1. Field of the Invention
The present invention relates to an information processing apparatus and, more particularly, to an information processing apparatus including an image capturing system including at least two cameras for capturing an external image of the real world, and a display system for superposing a CG (Computer Graphics) image such as 3D-CAD data on the image captured by the image capturing system, and displaying the obtained image.
2. Description of the Related Art
The MR (Mixed Reality) technique can exhibit an object drawn by CG or the like as if the object existed in the real world. The MR technique is implemented by superposing a CG image of a virtual world generated by CG on a captured image of the real world, and displaying a composite image. A video see-through type HMD (Head Mounted Display) is known as an image display apparatus for implementing the MR technique.
The video see-through type HMD includes an image capturing system for capturing an external image of the real world, a CG image generator for generating a CG image of a virtual world as if the image were seen from an image capturing position in the real world, a composite image generator for generating a composite image by compositing the external image and CG image, and a display system for displaying the composite image.
The image capturing system captures an external image of the real world in a state in which the image capturing direction is almost matched with the line of sight from the pupil position of the user, thereby obtaining a captured image. Note that in the image capturing system, right and left cameras are arranged such that their optical axes are almost parallel. A human observes an object by rotating the right and left eyes inward so that the optical axes of the two eyes intersect at the object. This rotating motion of the eye is called convergence (congestion). A captured image obtained by the image capturing system has no convergence, because the optical axes of the right and left cameras are almost parallel. To obtain a natural image by giving convergence to the captured image having no convergence, a process of extracting a specific range from the captured image is performed. This extraction process can provide an image in which the two optical axes of the image capturing system virtually intersect at an object, so the user can feel as if he or she were observing an external world with convergence. The HMD provides the user with a composite image obtained by superposing a CG image on a captured image extracted to have convergence. The HMD measures the position and posture of the main body, superposes a generated CG image on a captured image in accordance with the measured position and posture, and displays the composite image, thereby displaying the captured image of the real world and the CG image of a virtual world in a correct positional relationship. To display the captured image and CG image in the correct positional relationship, it is necessary to measure the camera parameters such as the relative positions, focal lengths, principal point positions, and distortions of the cameras of the image capturing system. In the image capturing system of the HMD, the positional relationship between the two cameras is fixed in order to fix the camera parameters. The HMD displays a natural image for the user by matching the optical axis of the display system with the optical axes of the two cameras of the image capturing system.
The pupil distances of humans have individual differences (in other words, humans do not all have the same distance between their pupils). Therefore, to reduce the influence of the individual difference of the HMD user on the impression of use, there is a method of adjusting the pupil distance of the HMD. When this pupil distance adjustment is performed, the image capturing system and display system can be arranged in positions matching the pupil distance of the user. This makes it possible to provide the HMD for more users.
Japanese Patent Laid-Open No. 2005-311754 has disclosed a method of matching the image capturing system and display system with the pupil distance of the user by simultaneously moving the two systems.
In Japanese Patent Laid-Open No. 2005-311754, however, the camera parameters such as the relative positional relationship between the right and left cameras change because the image capturing system moves simultaneously with the display system. This change in camera parameters caused by the movement of the image capturing system sometimes makes it impossible to correctly superpose a CG image on a captured image.
Also, convergence has individual differences because the pupil distances of the users are different. In Japanese Patent Laid-Open No. 2005-311754, the extraction range is not changed in accordance with the individual difference of convergence, so as to give convergence to a captured image. This sometimes makes it impossible to provide a captured image having convergence matching the user.